Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Data from: Bird responses to lowland rainforest conversion in Sumatran smallholder landscapes, Indonesia

Rapid land-use change in the tropics causes dramatic losses in biodiversity and associated functions. In Sumatra, Indonesia, lowland rainforest has mainly been transformed by smallholders into oil palm (Elaeis guineensis) and rubber (Hevea brasiliensis) monocultures, interspersed with jungle rubber (rubber agroforests) and a few forest remnants. In two regions of the Jambi province, we conducted point counts in 32 plots of four different land-use types (lowland rainforest, jungle rubber, rubber plantation and oil palm plantation) as well as in 16 nearby homegardens, representing a small-scale, traditional agricultural system. We analysed total bird abundance and bird abundance in feeding guilds, as well as species richness per point count visit, per plot, and per land-use system, to unveil the conservation importance and functional responses of birds in the different land-use types. In total, we identified 71 species from 24 families. Across the different land-use types, abundance did not significantly differ, but both species richness per visit and per plot were reduced in plantations. Feeding guild abundances between land-use types were variable, but homegardens were dominated by omnivores and granivores, and frugivorous birds were absent from monoculture rubber and oil palm. Jungle rubber played an important role in harbouring forest bird species and frugivores. Homegardens turned out to be of minor importance for conserving birds due to their low sizes, although collectively, they are used by many bird species. Changes in functional composition with land-use conversion may affect important ecosystem functions such as biological pest control, pollination, and seed dispersal. In conclusion, maintaining forest cover, including degraded forest and jungle rubber, is of utmost importance to the conservation of functional and taxonomic bird diversity

Homegarden plants

Plant species found during he homegarden survey. Abundances are approximate above 30 individuals

Bird data homegardens

Birds detected during 3-4 point count visits inside plots situated in homegardens of Jambi, Sumatra. The stratum and horizontal distance were not recorded in homegardens. The used plants were only recorded in homegardens. Plot code letters represent the landscape (B: Bukit 12, H: Harapan Rainforest), the land-use (F: forest, J: jungle rubber, R: rubber, O: oil palm, G: homeGarden), and the replicate

Bird survey info

Point count visit info with date, time, cloud cove

Homegardens data, anonymous

General homegarden data (management, vegetation, administrative

Point count analysis R script

All statistical models and figures can be reproduced with this script

Bird data core plots

Birds detected during 3-4 point count visits inside plots situated in forest, jungle rubber, rubber and oil palm plantations. Plot code letters represent the landscape (B: Bukit 12, H: Harapan Rainforest), the land-use (F: forest, J: jungle rubber, R: rubber, O: oil palm, G: homeGarden), and the replicate. Some lines with zero detections were included when the plot was visited but no bird was detected